Second Order Measures for Uncertainty Processing

Uncertainty processing methods are analysed from the viewpoint of their sensitivity to small variations of certainty factors. The analysis makes use of the algebraic theory which defines the function for combining partial certainty factors by means of a group operation of the ordered Abelian group over the interval of uncertainty. Two approaches are introduced: (a) sensitivity analysis of the inference network and (b) calculation of second order probabilities. Sensitivity functions are defined as partial derivatives of the combining function with respect to their arguments. Based on the sensitivity functions, we define the path sensitivity which measures the sensitivity of a larger part of the inference network. If a set of samples of certainty factors is available instead of a single value, the second order probability distribution can be approximated by the distribution of an average value. It is shown that the parametric form of the distribution is completely determined by the combining function. 1 I n t r oduc t i on Numerical values describing the uncertainty of knowledge and data in knowledge-based (KB) systems are usually imprecise due to the fact that they are almost always provided as the subjective assessments of experts or users. Nonetheless, these imprecise numbers are processed by some algorithm and the results are used to draw conclusions. Without a thorough KB verification which includes an analysis of the robustness of the uncertainty processing technique used, we must always be aware of die limited credibility of results. This paper aims to provide techniques for such an analysis. The methods described are based on compositional (extensional) calculation of uncertainty processing [Duda et id., 1976; Gashnig, 1980; Heckerman, 1986; Reiter, 1980; Wise, 1988] (see [Hajek et al., 1992; Pearl, 1988] for more detailed discussion). Although the current attention of the Al community is focused rather on intensional (model-based, probabilistic) approaches [Spiegelhalter, 1986; Lauritzen and Spiegelhalter, 1988; Pearl, 1988], the compositional methods are still popular due to their computational simplicity. The main objection to the compositional methods is that the results are not sound. In [Hajek et al., 1992] an attempt is made to revive these methods by replacing the original simple-minded interpretation of their results by a comparative one, thus improving their robustness as well as their soundness. We wi l l present two methods for assessing the imprecision of uncertainty measures in rule-based KB systems. The first approach is based on sensitivity evaluation. The idea of a sensitivity analysis of inference nets was explored in Prospector [Gashnig, 1980], where a uranium model was compiled and run for a large number of combinations of data and the sensitivity was calculated. Our approach is more analytical. We define sensitivity functions for particular methods of combining certainty factors and then in terms of these functions and rule sensitivities, we analyse the sensitivity of a path in the inference network. The second method proposed in this paper is based on the idea of second order uncertainties, i.e. the uncertainties of certainty factors. The concept of second order probabilities has already been suggested by [Cheeseman, 1985]. We wil l show that for certain statistics the parametric form of the second order probability density is completely determined by the method used for combining certainty factors. This property makes it possible to calculate the actual second order density functions. Moreover, the parametric form of this density function is invariant with respect to the combining function used.